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Abstract. We show that the Kolmogorov distance allows to quantify memory 
effects in classical stochastic processes by studying the evolution of the single- 
time probability distribution. We further investigate the relation between the 
Kolmogorov distance and other sufficient but not necessary signatures of non- 
Markovianity within the classical setting. 



1. Introduction 

In the theory of classical stochastic processes the definition of Markov process 
relies on a requirement involving the entire hierarchy of conditional probability 
distributions pQ. A non-Markov process is then defined to be a stochastic process 
whose conditional probabilities do not comply with this requirement. In particular, 
the transition probabilities of a Markov process obey the well-known Chapman- 
Kolmogorov equation, which in the homogeneous case can be read as a semigroup 
composition law for a family of transition operators defined in terms of the transition 
probabilities of the process [I] . A Markov process can be considered as a process that 
lacks memory, whereas non-Markov processes describe phenomena in which memory 
effects are relevant. 

Different approaches to the notion of stochastic process in quantum mechanics 
have been developed, based on C*-algebras [21 El 0] or on the stochastic Schrodinger 
equation [6j [7], just to mention some relevant examples, but there is not a well- 
established general theory of quantum stochastic processes. The crucial difference 
between the quantum and the classical setting traces back to the peculiar role played 
by measurement in the quantum case. In order to make statements about the values 
of observables of a quantum system at different times one has to specify measurement 
procedures, which modify the subsequent evolution [8]. The generic dynamics of a 
quantum system, also taking the interaction with its environment into account, i.e. 
the dynamics of an open quantum system [5] , is usually formulated by means of a one- 
parameter family of completely positive dynamical maps, which fixes the evolution of 
the statistical operator representing the open system's state. Within this framework, 
where a straightforward translation of the classical definition does not apply, the 
non-Markovianity of a quantum dynamics has recently been defined and quantified 
in terms of specific properties of the dynamical maps [TU1 HU HH H3]- In analogy 
with the transition operators of classical Markov processes, one can identify quantum 
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Markovianity with a proper composition law of the dynamical maps |12| , which reduces 
to the semigroup law in the time-homogeneous case. In a different and non-equivalent 
way [141 115] . one can identify quantum Markovianity with the contractivity of the 
trace distance between states of the open quantum system during the dynamics [11] . 
The trace distance quantifies the distinguishability of quantum states [16] , so that its 
variations in the course of time can be read in terms of an information flow between 
the open system and its environment [TTJ [TU [13] . An increase of the trace distance 
indicates that the information is flowing from the environment to the open system, so 
that the influence the open system has had on the environment affects the open system 
back again. The trace distance allows to formulate the physical idea of memory effects 
as typical of non-Markovian dynamics in a mathematically definite way. 

The precise relation between the definition of Markov stochastic process and the 
different notions of quantum Markovianity has been investigated in JXT1 [18]. More 
specifically, one can introduce also at the classical level a family of dynamical maps 
which describes the evolution of the single-time probability distribution of a stochastic 
process. The properties used to define the Markovianity of quantum dynamics 
naturally induce signatures of non-Markovianity for the classical stochastic processes. 
The evolution of the single-time probability distribution does not allow to assess the 
Markovianity of a stochastic process. However, the violation of either of the above- 
mentioned properties provides a sufficient condition for the stochastic process to be 
non-Markovian pTl [18]. In this paper, we deepen the analysis of the signatures of 
non-Markovianity and our entire discussion will be kept within the classical setting. 
We first focus on the signature of non-Markovianity based on the evolution of the 
distinguishability between single-time probability distributions, which is quantified 
through the Kolmogorov distance, i.e., the classical counterpart of the trace distance. 
In particular, we show that the increase in time of the Kolmogorov distance provides a 
quantitative description of the memory effects present in the evolution of single-time 
probability distributions. We take into account two-site semi-Markov processes [T], 
which allow for a complete characterization in terms of a waiting time distribution. 
By means of the Kolmogorov distance, we study how the different features of the 
waiting time distribution influence the memory effects in the subsequent evolution 
and then the possibility to detect the non-Markovianity of the process at the level of 
single-time probability distributions. Furthermore, we investigate the relation between 
the Kolmogorov distance and the differential equation satisfied by the single-time 
probability distribution. The latter is strictly related to the composition law of the 
classical dynamical maps and it provides a further signature of non-Markovianity, 
which is shown not to be equivalent to the increase of the Kolmogorov distance. 



2. Memory effects in semi-Markov processes 

For the sake of simplicity, let us consider a stochastic process taking values in a finite 
set. The single-time probability distribution at time t is then a probability vector p (t), 
so that its elements denoted as Pfc(i) k = 1, . . . N, satisfy pk{t) > and ^2 k Pk(t) = 1- 
As in the quantum setting [9] , one can describe the time evolution of p (t) by means 
of a one-parameter family of dynamical maps {A(i, 0)} 4> q 

p(t) =A(t,0)p(0), (1) 
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with t = fixed initial time. It is easy to see that a matrix A associates probability 
vectors to probability vectors if and only if its entries (A)jj. satisfy the conditions: 

N 

(A) jk >0 £(A) jfc = l, (2) 
i=i 

Vj, k = 1, ...,N. A matrix which fulfills equation ([2]) is usually called stochastic 
matrix. We will thus require that every dynamical map A(t, 0) is a stochastic matrix. 

As long as only the single-time probability distribution is taken into account, 
one cannot in general determine whether the stochastic process is Markovian or 
not. Nevertheless, one can introduce signatures of non-Markovianity at the level 
of the evolution of single-time probability distributions. These can be understood 
as sufficient but not necessary conditions for the process to be non-Markovian. The 
Kolmogorov distance Dk {p 1 ,p 2 ) between two probability distributions p 1 and p 2 on 
a common finite set X is defined as 

Dk(p\p 2 ) =W\PI-Pl\- ( 3 ) 

kex 

Now, consider the time evolution of the Kolmogorov distance D k (p 1 (t) , p 2 (t)) 
between probability vectors evolved from different initial conditions through 
the stochastic matrix in ([TJ. The Chapman-Kolmogorov equation implies (TT] 
Dk (p 1 (t) , p 2 (t)) < Dk (p 1 (s),P 2 (s)), Vt > s, so that any revival in the evolution 
of the Kolmogorov distance is a signature of the non-Markovianity of the process. By 
adapting the interpretation introduced for the dynamics of open quantum systems 
[ITJ [13] , any increase of the Kolmogorov distance can be read as a memory effect in 
the evolution of the single-time probability distribution. The information about the 
initial condition that is contained in the initial value of the Kolmogorov distance can be 
partially lost and then recovered, implying an increase of the distinguishability between 
the probability distributions evolved from the two different initial conditions. We can 
quantify the memory effects in the overall dynamics fixed by the stochastic matrices 
A(t,0) by integrating the rate of change of the Kolmogorov distance er(t,p 1,2 (0)) = 
(d/dt)Dx (p 1 (t) , p 2 (t)) over the time-intervals where the information about the initial 
condition is recovered. Namely, we define (compare with [TT]) 



Af c (A)= max I dt a(t,p^ 2 (0)), (4) 



'(T>0 

where the maximum among the couples of initial probability vectors is taken. A/"c(A) 
points out the relevance of the signature of non-Markovianity in the evolution of single- 
time probability distributions given by the increase of the Kolmogorov distance. 

As a specific example, let us consider the semi-Markov processes [TJ, which are 
a class of stochastic processes allowing for a compact characterization. The semi- 
Markov processes combine features of Markov chains and renewal processes [TJ5] : they 
describe a system moving among different sites in a way such that the random times 
separating the different transitions as well as the transition probabilities between the 
different sites only depend on departure and arrival sites. A semi-Markov process 
is completely determined by the semi-Markov matrix Q(t), whose entries (Q)jk(t) 
are the probability densities to make the jump k — > j in a time t. For the sake of 
simplicity, we will take into account a system moving between two different sites and a 

semi-Markov matrix Q (t) = f ^ Q j f(t)- This means that once in a site the system 
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jumps with certainty to the other, with a random time before each new jump given 
by the site- independent waiting time distribution f(i). The semi-Markov process will 
be Markovian according to the precise definition of the classical stochastic processes 
if and only if the waiting time distribution is the exponential one |20j 

g{t) = \e- xt . (5) 

For a generic waiting time distribution, the dynamical maps A(i, 0), which can be 
obtained through an integrodifferential equation fixed by the semi-Markov matrix 
[17] . are 

Mt o) = l -( 1 + q{t) x ~ q ^ ) (6) 

Ht ' U) 2 { l-q(t) l + q(t) J' (b) 

Here, q (t) = ^2^L p(2n,t) ~ Y^=oPi^ n + expresses the difference between the 
probability to have an even or an odd number of jumps up to time t. This quantity is 
related to the waiting time distribution via the corresponding Laplace transforms 

1 1 -f(u) 

?« = -: jtt> ( 7 

«l + /(u) 

where v(u) = J °° dt v(t) e~ ut is the Laplace transform of the function v(t). The 
dynamical map A(i, 0) associates the initial probability vector to the probability 
vector at time t and then determines the expression of the Kolmogorov distance. 
In particular, for A(t, 0) as in (|6]) one has 

Dk (p^p 2 ®) = W)\D K (p\Q),p 2 (fi)) . (8) 

Thus, the memory effects in the evolution of the single-time probability distribution 
are quantified by, see (@|, 

Afc(A)= [ dt±\q(t)\ = J2 (l#i)|-|?(«0l), (9) 

where f2+ = |Jj( a i) ^i) corresponds to the time intervals in which \q(t)\ increases. The 
maximum growth of the Kolmogorov distance in the course of time is obtained for the 
two initial probability vectors (1,0) T and (0, 1) T . 

By studying the quantity A/c(A) in ^ for different choices of the waiting 
time distribution f(t), we can describe how different semi-Markov processes are 
characterized by a different amount of memory effects in the evolution of p(t). An 
interesting class of waiting time distributions is given by the so-called special Erlang 
distributions of order n. The latter consist in the convolution of n equal exponential 
waiting time distributions of parameter A, see ([5]) : 

= (10) 

a 

A special Erlang distribution of order n describes a random variable given by the 
sum of n independent and identically distributed exponential random variables. This 
waiting time distribution can be related to a system that jumps from one site to 
another in various unobserved stages. Each stage is exponentially distributed in time, 
but one observes only when n stages have occurred. In this situation, the non- 
Markovianity expressed by the non-exponential waiting time distribution is due to 
the lack of information about all the intermediate stages [20] . Indeed, one expects 
that the higher is the number of unobserved stages the stronger is the deviation 
from the Markovian situation. Now, A/a (A) allows to formulate this statement in 
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Figure 1. Size of memory effects in the evolution of p(t) for a two-site semi- 
Markov process versus the number n of the exponential distributions in the special 
Erlang waiting time distribution l llOU that fixes the process. The memory effects 
are quantified by means of A/"c(A) defined in {5}, where q(t) is obtained through 
equation J7J). For n = 2, A/c(A) = 0.045 (compare with figure 



a precise way since it quantifies the memory effects in the evolution of p(t), which 
represent a signature of non-Markovianity. In figure I, we plot Ac? (A) for a semi- 
Markov process that describes a system moving between two different sites with a 
waiting time distribution as in ((10)) . For /(£) = Ae~ At the Kolmogorov distance 
is monotonically decreasing [17], so that A/"c(A) = 0. Then, A/a (A) increases with 
the number of exponential distributions in (|10j) and, quite remarkably, it grows to a 
good approximation in a linear way for n > 3. The memory introduced by the lack of 
information about intermediate stages growths linearly with the number of unobserved 
stages. In addition, let us note that A/c(A) does not depend on the parameter A in 
(|10[) . By the properties of the Laplace transform or by dimensional analysis, one can 
easily find that q(t) is a function of Xt. Maxima and minima of q(t) do not depend 
on A and thus neither does A/"c(A), see the right hand side of ((0J. Only the functional 
expression of the waiting time distribution matters. 

As a further example, we consider the convolution of two non-exponential waiting 
time distributions. Coming back to the picture of a system jumping between two 
sites with unobserved stages, we study how the memory in the evolution of p(t) is 
influenced by the specific waiting time distribution that characterizes the intermediate 
stages. Let the waiting time distribution /(£) be the convolution of two equal 
mixtures of exponential distributions with rates, respectively, Ai and A2: given 
h(t) = /iA,e~ Alt + (1 - /i)A 2 e- A2t , we set 



A mixture ^ fc Hkfkit) describes a system that moves between different sites following 
with probability /itfe the distribution /&(£). Thus, f(t) as in fTT)) means that both the 
unobserved stages are exponentially distributed in time with a rate that is given by Ai, 
with probability fx, or by A2, with probability 1 — 11. In figure^ we plot the evolution of 
|<f(t)|, see ((7|), for /(£) as in (JTTJ). Recall that \q(t)\ is the Kolmogorov distance for the 
two initial probability vectors that maximize Ac (A), see ©. Moreover, if the waiting 
time distribution is simply the mixture of two exponential distributions, then the 
Kolmogorov distance is monotonically decreasing [T7], so that Ac? (A) = 0. We can see 
in figure (2) that this is still the case for the convolution of two mixtures with balanced 
exponential distributions, i.e., [i = 1/2. Even more, only a strong imbalance between 



f(t) = (h*h)(t). 



(11) 
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Figure 2. Plot of \q(t)\ versus time for a waiting time distribution given by 
the convolution of two equal mixtures of exponential distributions, sec lllljl . for 
different values of the mixing parameter /i; in addition, A2/A1 = 5. A/"c(A) in $9$ 
is equal to for fj, = 0.5 and fj, = 0.2, while A/"c(A) = 0.022 for /j,=0.1. 



the weights of the two exponential distributions provides an increase of the Kolmogorov 
distance, witnessing the presence of memory effects. In any case, this signature of non- 
Markovianity is smaller than for the convolution of two equal exponential distributions, 
i.e., for f(t) as in (fTQ|) with n = 2, which is recovered from for fx = or /j, = 1. If 
the rate of the exponential distributions of the unobserved stages is not known with 
certainty, the memory due to the lack of information about the occurrence of the 
stages is reduced. In particular, if the available exponential distributions are equally 
probable, there will be no memory effects at all. 



3. Kolmogorov distance and time-local equation 

In this section, we study the relation between the Kolmogorov distance and the 
differential equation satisfied by p(t) 7 which provides further signatures of non- 
Markovianity. From the knowledge of the dynamical maps A(t, 0), one can directly 
obtain a time-local equation in the form 

Ap( t ) = L(t)p(t), (12) 

by means of the relation [3TJ [52] 

L(t) = ^MA-^,0). (13) 

In particular, consider the case such that L(t) satisfies the so-called Kolmogorov 
conditions for any t > 0, i.e., (L(4))jfc > for j ^ fc, (L(t))jj < and X)fc(^(^))jfe = 
for any j. Equation (|12[) is then equivalent to the system of equations 

= £0*k (t)Pk(t) - W kj (t) Pj (t)) (14) 
fc 

with W jk (t) > linked to (L{t)) jk through (L(t)) jk = W jk {t) - *£t W lk (t)6 jk . One 
can show [53] that L(t) satisfies the Kolmogorov conditions for any t > if and only 
if the corresponding family of dynamical maps {A(t, 0)} t>0 satisfies 

A(t, 0) = A(t, s)A(s, 0) (15) 
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for any t > s > 0, with A(t, s) itself being a stochastic matrix. The composition 
law (fT5"]) . named P-divisibility in [T7], represents the classical counterpart of the 
composition law exploited in |12| to define the notion of quantum Markovianity 
[171 118] . The violation of P-divisibility provides a further signature of the non- 
Markovianity of classical stochastic processes at the level of single-time probability 
distributions. Note that this signature of non-Markovianity can be directly read from 
the time- local equation (Q~4|) since P-divisibility is violated if and only if Wjk(t) < 
for some j, k and t > 0. As emphasized [9] [24] for the quantum dynamics of statistical 
operators, also the evolution of the single-time probability distribution of a classical 
stochastic process can be described through a time-local equation even if the process 
is non-Markovian. In particular, negative coefficients in the equation will account for 
memory effects in the evolution of p(t) [25] . 

For the two-site semi-Markov processes described in the previous section, with 
A(t, 0) as in one has 

with 

dq(t)/dt 

l{t) = 2qW' (17) 
Thus, the elements pj (t), j = 1, 2, satisfy the following system of differential equations: 

^Pi(*)=7(*)(P2(*)-Pi(*)) 

^P2(*)=7(*)(Pi(*)-P2W)- (18) 
The rate ^(t) in (fT5|) is positive if and only if the corresponding Kolmogorov distance 
is monotonically decreasing, see ([8]) and (fT7)) . For these processes the increase of 
the Kolmogorov distance and the negativity of the coefficients in the time local 
equation (|14[) . or, equivalently, the violation of P-divisibility, provide signatures of 
non-Markovianity that arc fully equivalent. One could show that this is the case for 
any two-site semi-Markov process with a site-independent waiting time distribution. 

Now, consider any non-Markov process whose single-time probability distribution 
satisfies 

^Pi(*)=72(t)Pa(*)-7i(*)l>i(f) 

d p2(t) =7i(*)Pi(f) -Tia(*)P2(*), (19) 



dt 



with 



7 2 (i)<0 for some t>0; ji(t)>0 Vt>0 

7i(t)+Ti(*)>0 Vi>0. (20) 

If we further ask dr72(r) > for any t > the resulting dynamics is well-defined, 
i.e. the positivity of p(t) is conserved. By solving (|T9| . one easily finds 

Dk (p\t) lP \t)) = e-f: d ^ + ^D K {pH0),p 2 (0)) , (21) 
which is a monotonically decreasing function because of the last condition in (1201) . 
Thus, there are processes such that the Kolmogorov distance Dk (p 1 (t) , p 2 (t)) does 
not increase in time, even if there is some negative coefficient in the time-local equation 
(|14[) . see the first condition in (|2T)|) . so that P-divisibility is violated: these signatures 
of non-Markovianity are in general not equivalent. 
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4. Conclusions 

Wc have investigated signatures of non-Markovianity in the evolution of the single-time 
probability distribution p(t) of classical stochastic processes. We have first focused on 
the increase of the Kolmogorov distance between probability vectors. This allows to 
detect memory effects related to the initial condition in the subsequent evolution of 
p(t) and it naturally leads to the introduction of a quantity that measures the relevance 
of the memory effects, which can be seen as the classical counterpart of the measure 
for quantum non-Markovianity introduced in jllj . We have then discussed the relation 
between the increase of the Kolmogorov distance and the time-local equation satisfied 
by the probability vector. Wc have shown how for two-site semi-Markov processes 
with a site-independent waiting time distribution the negativity of the rates in the 
time-local equation is equivalent to a non-monotonic evolution of the Kolmogorov 
distance, further presenting examples for which this is not the case. 
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